Item recommendations can be generated for customers of electronic commerce sites in order to suggest additional items that may be of interest to a particular user based on an item that the user may be viewing. Yet, some recommended items suggested to a user may not be relevant to the item in terms of style, color, or other aspects of the item. Additionally, the use of an expansive catalog of items can make identifying items that are similar to a particular item on behalf of a user difficult or problematic.